Analysing Cryptocurrency Market
Nisha Selvarajan
- Challenge
- Data Description
- Data Analysis
- Prepare Data for Regression
- Linear Regression To Predict Market Price
- Residual Analysis in Linear Regression
- Linear Regression Prediction & Accuracy.
- Polynominal Regression
- Polynomial Regression Prediction & Accuracy
- Spline Regression
- Spline Regression Prediction & Accuracy
- Generalized Linear Model
- GAM Regression Prediction & Accuracy
- Scatter Plot 3D Visualization
- Penalized Cubic Regression Spline
- Cubic Regression Spline Prediction & Accuracy
- Extreme Gradient Boosting
- Extreme Gradient Boosting Prediction & Accuracy
- Compare Regression Models
- Conclusion
Challenge
To forecast cryptocurrency prices using all the trading features like price, volume, open, high, low values present in the dataset.
Probably one of the biggest things in recent years is Bitcoin. Bitcoin grew by around 800% last year, held a market cap of around 250 billion dollars, and sparked worldwide interest in cryptocurrencies. But what are cryptocurrencies? Basically they’re digital currencies that use complex computer algorithms and encryption to generate more currency and to protect transactions. What’s really cool about cryptocurrencies is that they utilize a network of thousands of computers that forward people’s transactions to what’s known as a blockchain (essentially a big record of transactions kept secure by the network of computers). Once a transaction is in the blockchain, it’s never coming out again; this protects cryptocurrencies from double-spends. So it’s pretty clear that cryptocurrencies are a cool new way to spend money — what if we could predict how its prices fluctuate?
By analyzing bit coin historical features, such as bitcoin tradevolume, bitcoin blockssize, bitcoin difficultyto find a new block,total value of coinbase block rewards , transaction fees paid to miners, we can predict the variation and can predict the ups and downsof bitcoin price .
Data Description
The data used for this particular project is “Cryptocurrency Historical Prices by Sudalairajkumar”. The data consists of information on the 17 currencies collected daily from various dates (Some January 2016, April 2013, August 2015, so on.) to October 2017. The original source of the data is coinmarketcap, Blockchain Info and Etherscan. (https://www.kaggle.com/sudalairajkumar/cryptocurrencypricehistory)
We are using Bitcoin Dataset (bitcoin_dataset.csv) in the above dataset.The details of attributes are following.
| Names | Description |
|---|---|
| Date | Date - Date of observation |
| btcmarketprice | Numerical - Average USD market price across major bitcoin exchanges |
| btctotalbitcoins | Numerical - Total number of bitcoins that have already been mined |
| btcmarketcap | Numerical - Total USD value of bitcoin supply in circulation |
| btctradevolume | Numerical - Total USD value of trading volume on major bitcoin exchanges |
| btcblockssize | Numerical - Total size of all block headers and transactions |
| btcavgblock_size | Numerical - Average block size in MB |
| btcnorphaned_blocks | Numerical - Total number of blocks mined but ultimately not attached to blockchain |
| btcntransactionsperblock | Numerical - Average number of transactions per block |
| btcmedianconfirmation_time | Numerical - Median time for a transaction to be accepted into a mined block |
| btchashrate | Numerical - Estimated number of tera hashes per second the Bitcoin network is performing |
| btc_difficulty | Numerical - Relative measure of how difficult it is to find a new block |
| btcminersrevenue | Numerical - Total value of coinbase block rewards and transaction fees paid to miners |
| btctransactionfees | Numerical - Total value of all transaction fees paid to miners. |
| btccostpertransactionpercent | Numerical - Miners revenue as percentage of the transaction volume. |
| btccostper_transaction | Numerical - Miners revenue divided by the number of transactions |
| btcnunique_addresses | Numerical - Total number of unique addresses used on the Bitcoin blockchain. |
| btcntransactions | Numerical - Number of daily confirmed Bitcoin transactions |
| btcntransactions_total | Numerical- Total number of transactions |
| btcntransactionsexcludingpopular | Numerical- Total number of Bitcoin transactions, excluding the 100 most popular addresses |
| btcntransactionsexcludingchainslongerthan_100 | Numerical- Total number of Bitcoin transactions per day excluding long transaction chains |
| btcoutputvolume | Numerical- Total value of all transaction outputs per day |
| btcestimatedtransaction_volume | Numerical- Total estimated value of transactions on the Bitcoin blockchain |
| btcestimatedtransactionvolumeusd | Numerical- Estimated transaction value in USD value |
Data Analysis
- Plot missing values of all the features in the dataset.
- Ploting histograms for numerical variables.
- Get metric table with many indicators for all numerical variables, automatically skipping the non-numerical variables.
## variable mean
## 1 btc_market_price 1.156936e+03
## 2 btc_total_bitcoins 1.335988e+07
## 3 btc_market_cap 1.862023e+10
## 4 btc_trade_volume 1.056425e+08
## 5 btc_blocks_size 4.644976e+04
## 6 btc_avg_block_size 4.558738e-01
## 7 btc_n_orphaned_blocks 4.596988e-01
## 8 btc_n_transactions_per_block 8.670633e+02
## 9 btc_median_confirmation_time 9.706814e+00
## 10 btc_hash_rate 1.793447e+06
## 11 btc_difficulty 2.335498e+11
## 12 btc_miners_revenue 2.953228e+06
## 13 btc_transaction_fees 7.745935e+01
## 14 btc_cost_per_transaction_percent 2.431859e+00
## 15 btc_cost_per_transaction 1.812491e+01
## 16 btc_n_unique_addresses 2.507467e+05
## 17 btc_n_transactions 1.317519e+05
## 18 btc_n_transactions_total 9.027881e+07
## 19 btc_n_transactions_excluding_popular 1.218026e+05
## 20 btc_n_transactions_excluding_chains_longer_than_100 8.148942e+04
## 21 btc_output_volume 1.901380e+06
## 22 btc_estimated_transaction_volume 2.322481e+05
## 23 btc_estimated_transaction_volume_usd 2.734628e+08
## 24 Days 1.770500e+03
## std_dev variation_coef p_01 p_05 p_25
## 1 2.674346e+03 2.3115755 4.154962e+00 5.159834e+00 1.078087e+02
## 2 2.576863e+06 0.1928807 7.950809e+06 8.632810e+06 1.130566e+07
## 3 4.497526e+10 2.4153979 3.305561e+07 4.602839e+07 1.219861e+09
## 4 3.496546e+08 3.3097915 1.248379e+05 2.511911e+05 3.828257e+06
## 5 4.566377e+04 0.9830788 6.181400e+02 8.577000e+02 8.477250e+03
## 6 3.454998e-01 0.7578847 1.646062e-02 2.382558e-02 1.452091e-01
## 7 9.225268e-01 2.0068068 0.000000e+00 0.000000e+00 0.000000e+00
## 8 6.730492e+02 0.7762400 3.600000e+01 5.100000e+01 3.042500e+02
## 9 3.280905e+00 0.3380002 5.633333e+00 6.266667e+00 7.487500e+00
## 10 3.746989e+06 2.0892672 8.496842e+00 1.075662e+01 1.458127e+02
## 11 4.764777e+11 2.0401546 1.155038e+06 1.498294e+06 1.933926e+07
## 12 6.460895e+06 2.1877402 2.852060e+04 3.945052e+04 4.705878e+05
## 13 1.288653e+02 1.6636505 3.490310e+00 6.457239e+00 1.457394e+01
## 14 1.901754e+00 0.7820164 4.825175e-01 6.451919e-01 1.044436e+00
## 15 2.357916e+01 1.3009257 1.056201e+00 1.820013e+00 5.846544e+00
## 16 2.080185e+05 0.8295965 9.457350e+03 1.230940e+04 5.769100e+04
## 17 1.011909e+05 0.7680411 5.598360e+03 7.162900e+03 5.163900e+04
## 18 8.654108e+07 0.9585979 2.084944e+06 2.681650e+06 1.962744e+07
## 19 1.039818e+05 0.8536909 4.074090e+03 5.375700e+03 2.626100e+04
## 20 6.993201e+04 0.8581729 5.219570e+03 6.405250e+03 2.473150e+04
## 21 2.438478e+06 1.2824783 3.185585e+05 4.395482e+05 8.253261e+05
## 22 2.376585e+05 1.0232961 5.774836e+04 8.201880e+04 1.492490e+05
## 23 6.587933e+08 2.4090782 4.552131e+05 7.998189e+05 1.893800e+07
## 24 6.519728e+02 0.3682422 6.645700e+02 7.548500e+02 1.206250e+03
## p_50 p_75 p_95 p_99 skewness kurtosis
## 1 3.814600e+02 6.772899e+02 6.113308e+03 1.513347e+04 4.0204186 20.208014
## 2 1.373259e+07 1.579800e+07 1.665550e+07 1.683308e+07 -0.4249548 2.022534
## 3 5.353668e+09 1.016825e+10 1.021500e+11 2.534300e+11 4.0331240 20.258732
## 4 1.619585e+07 4.004024e+07 5.424950e+08 1.795404e+09 6.5348233 60.776654
## 5 2.866300e+04 7.801137e+04 1.391994e+05 1.540718e+05 0.8882639 2.523000
## 6 3.532110e-01 7.937469e-01 1.014817e+00 1.063274e+00 0.3924264 1.627850
## 7 0.000000e+00 1.000000e+00 2.000000e+00 4.000000e+00 2.3975887 9.228294
## 8 6.255000e+02 1.467960e+03 2.076422e+03 2.290601e+03 0.5989092 2.008831
## 9 8.633333e+00 1.131667e+01 1.583583e+01 2.027150e+01 2.1363472 13.885025
## 10 3.068903e+05 1.571067e+06 9.968082e+06 1.986087e+07 3.2782429 14.839627
## 11 4.127287e+10 2.130000e+11 1.350000e+12 2.600000e+12 3.1601083 13.984572
## 12 1.243021e+06 2.249654e+06 1.308094e+07 4.025868e+07 4.6169422 26.617086
## 13 3.061101e+01 6.516377e+01 3.275609e+02 6.649762e+02 3.6877789 21.521243
## 14 1.743262e+00 3.270974e+00 6.298300e+00 9.093330e+00 1.6469959 6.094625
## 15 8.639663e+00 2.115561e+01 6.159131e+01 1.228364e+02 2.8345181 12.239496
## 16 1.969400e+05 4.082422e+05 6.064957e+05 8.683416e+05 0.7822109 2.967797
## 17 9.002350e+04 2.216978e+05 3.151295e+05 3.688130e+05 0.6628471 2.243549
## 18 5.701550e+07 1.472292e+08 2.672117e+08 2.961156e+08 0.9169656 2.596521
## 19 8.391800e+04 2.161795e+05 3.050850e+05 3.558842e+05 0.5973038 2.066496
## 20 4.881250e+04 1.391912e+05 2.108529e+05 2.463074e+05 0.7841609 2.341201
## 21 1.417049e+06 2.254076e+06 4.138097e+06 1.121818e+07 8.6344292 113.635350
## 22 2.090740e+05 2.765258e+05 4.059257e+05 5.508333e+05 15.1512301 298.006919
## 23 6.114545e+07 1.772323e+08 1.387115e+09 3.670032e+09 4.4027907 25.235036
## 24 1.770500e+03 2.334750e+03 2.786150e+03 2.876430e+03 0.0000000 1.800000
## iqr range_98 range_80
## 1 5.694811e+02 [4.1549616, 15133.4657326] [7.836673, 2659.848803]
## 2 4.492341e+06 [7950809, 16833083] [9458365, 16438351.25]
## 3 8.948388e+09 [33055607.5179, 2.5343e+11] [74040046.871, 43625054108.7]
## 4 3.621198e+07 [124837.85616, 1795404418.43] [378879.64888, 176935221.49]
## 5 6.953412e+04 [618.14, 154071.771914] [1821.7, 123564.74308]
## 6 6.485379e-01 [0.01646061914, 1.06327410774] [0.0805532014, 0.9719768372]
## 7 1.000000e+00 [0, 4] [0, 2]
## 8 1.163710e+03 [36, 2290.6014481] [183, 1895.7315191]
## 9 3.829167e+00 [5.633333333, 20.2715] [6.733333333, 14.01666667]
## 10 1.570921e+06 [8.49684165987, 19860869.4658999] [13.90415045, 5539841.6282]
## 11 2.129807e+11 [1155038.334, 2.6e+12] [1751454.535, 7.12e+11]
## 12 1.779066e+06 [28520.6011223, 40258679.6725] [49403.55884, 5753362.8584]
## 13 5.058982e+01 [3.4903098398, 664.976218459999] [10.905405781, 217.39422002]
## 14 2.226538e+00 [0.48251753199, 9.09332996994999] [0.7489782871, 5.0801821246]
## 15 1.530906e+01 [1.05620133014, 122.836372308] [2.7085213729, 44.071370197]
## 16 3.505512e+05 [9457.35, 868341.61] [26482.8, 540912.9]
## 17 1.700588e+05 [5598.36, 368813] [27340.2, 282147.6]
## 18 1.276018e+08 [2084943.99, 296115558.68] [4962182.2, 238282496.6]
## 19 1.899185e+05 [4074.09, 355884.24] [10596.2, 275570.1]
## 20 1.144598e+05 [5219.57, 246307.4] [11362.4, 191185]
## 21 1.428750e+06 [318558.514215, 11218178.2163999] [528344.8656, 3231172.4742]
## 22 1.272768e+05 [57748.36, 550833.32] [102573.2, 345053.54553]
## 23 1.582943e+08 [455213.09, 3670031860.53] [1584638, 715790079.46]
## 24 1.128500e+03 [664.57, 2876.43] [867.7, 2673.3]- Plot of daily bitcoin priceshows that this plot has a general upward trend with no obvious seasonality or intervention point. There is obvious changing variance. Successive observations suggest autoregressive behavior.
- Plot of market price with market capitalization.There is a linear trend for market cap with respect to market price.
- Correlation Plot to show the dependence between multiple variables in bitcoin dataset.Highlighting the most correlated variables in a data table. In this visual, correlation coefficients are also displayed.
## 'data.frame': 2258 obs. of 11 variables:
## $ btc_market_price : num 3.14 3.13 2.99 2.93 3.05 ...
## $ btc_total_bitcoins : num 7787350 7794850 7801700 7809700 7817650 ...
## $ btc_market_cap : num 24436704 24397803 23327083 22882421 23843832 ...
## $ btc_trade_volume : num 181505 363126 263375 90500 170165 ...
## $ btc_blocks_size : num 572 574 576 578 580 583 585 587 589 591 ...
## $ btc_hash_rate : num 8.51 8.13 7.43 8.68 8.62 ...
## $ btc_difficulty : num 1090716 1090716 1090716 1090716 1090716 ...
## $ btc_miners_revenue : num 24646 23487 20490 23449 24257 ...
## $ btc_transaction_fees : num 4.3 4.09 3.51 3.62 3.5 ...
## $ btc_cost_per_transaction : num 3.9 4.24 3.87 3.69 3.45 ...
## $ btc_estimated_transaction_volume_usd: num 10383421 11525011 9581607 5518235 17766452 ...
- Sometimes it’s nice to quickly visualise the data that went into a simple linear regression, especially when you are performing lots of tests at once. Here is a quick and dirty solution with ggplot2 to create the following regression plot with respect to btc_market_price:
Prepare Data for Regression
- Select variables relevant to market price:Based on the variable importance, we will use btc_market_cap, btc_hash_rate,btc_difficulty, btc_miners_revenue, btc_estimated_transaction_volume_usd feature for further analysis.
## 'data.frame': 2258 obs. of 6 variables:
## $ Market Price : num 3.14 3.13 2.99 2.93 3.05 ...
## $ Market Cap : num 24436704 24397803 23327083 22882421 23843832 ...
## $ Hash Rate : num 23 14 6 29 26 12 17 15 23 10 ...
## $ Difficulty : num 1090716 1090716 1090716 1090716 1090716 ...
## $ Miners Revenue : num 24646 23487 20490 23449 24257 ...
## $ Estimated Transaction Volume USD: num 10383421 11525011 9581607 5518235 17766452 ...- Load the cleaned dataset: - Omit rows which has N/A.
- Data slicing:
- Dataset is split into 80 percent of training data, 20 % of test set.
Linear Regression To Predict Market Price
- TrainingParameters :
- train() method is passed with repeated cross-validation resampling method for 10 number of resampling iterations repeated for 3 times.
- RMSE or Root Mean Squared Error is the default metrics used to evaluate algorithms on regression datasets in caret.RMSE or Root Mean Squared Error is the average deviation of the predictions from the observations.
# setup cross validation and control parameters
metric <- "RMSE"
tuneLength <- 10
# Training process
# Fit / train a Linear Regression model to dataset
linearModelReg <- caret::train(btc_market_price~
btc_market_cap+btc_hash_rate+
btc_difficulty+btc_miners_revenue+
btc_estimated_transaction_volume_usd
,data=subTrain1, method="lm", metric=metric,
preProc=c("center", "scale"), trControl=control, tuneLength = tuneLength)## + Fold1.Rep1: intercept=TRUE
## - Fold1.Rep1: intercept=TRUE
## + Fold2.Rep1: intercept=TRUE
## - Fold2.Rep1: intercept=TRUE
## + Fold3.Rep1: intercept=TRUE
## - Fold3.Rep1: intercept=TRUE
## + Fold1.Rep2: intercept=TRUE
## - Fold1.Rep2: intercept=TRUE
## + Fold2.Rep2: intercept=TRUE
## - Fold2.Rep2: intercept=TRUE
## + Fold3.Rep2: intercept=TRUE
## - Fold3.Rep2: intercept=TRUE
## + Fold1.Rep3: intercept=TRUE
## - Fold1.Rep3: intercept=TRUE
## + Fold2.Rep3: intercept=TRUE
## - Fold2.Rep3: intercept=TRUE
## + Fold3.Rep3: intercept=TRUE
## - Fold3.Rep3: intercept=TRUE
## Aggregating results
## Fitting final model on full training setsummary(linearModelReg)##
## Call:
## lm(formula = .outcome ~ ., data = dat)
##
## Residuals:
## Min 1Q Median 3Q Max
## -462.15 -28.95 -10.49 14.14 204.46
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1138.6559 0.5804 1961.911 < 2e-16 ***
## btc_market_cap 2422.4909 6.4585 375.086 < 2e-16 ***
## btc_hash_rate -126.0507 5.0542 -24.940 < 2e-16 ***
## btc_difficulty 145.3881 5.7482 25.293 < 2e-16 ***
## btc_miners_revenue 201.3575 4.9855 40.389 < 2e-16 ***
## btc_estimated_transaction_volume_usd -9.2744 2.1326 -4.349 1.39e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 42.73 on 5415 degrees of freedom
## Multiple R-squared: 0.9997, Adjusted R-squared: 0.9997
## F-statistic: 4.101e+06 on 5 and 5415 DF, p-value: < 2.2e-16Residual Analysis in Linear Regression
Linear regression is a statistical method for for modelling the linear relationship between a dependent variable y (i.e. the one we want to predict) and one or more explanatory or independent variables(X).
This residual plot will explain how residual plots generated by the regression function can be used to validate that some of the assumptions that are made about the dataset indicating it is suitable for a linear regression are met.There are a number of assupmtions we made about the data and these must be met for a linear model to work successfully and the standard residual plots can help validate some of these. These are:
The dataset must have some linear relationship
Multivariate normality - the dataset variables must be statistically Normally Distributed (i.e. resembling a Bell Curve)
It must have no or little multicollinearity - this means the independent variables must not be too highly correlated with each other. This can be tested with a Correlation matrix and other tests
No auto-correlation - Autocorrelation occurs when the residuals are not independent from each other. For instance, this typically occurs in stock prices, where the price is not independent from the previous price.
Homoscedasticity - meaning that the residuals are equally distributed across the regression line i.e. above and below the regression line and the variance of the residuals should be the same for all predicted scores along the regression line.
Four standard plots can be accessed using the plot() function with the fit variable once the model is generated. These can be used to show if there are problems with the dataset and the model produced that need to be considered in looking at the validity of the model. These are:
Residuals vs Fitted Plot
Normal Q–Q (quantile-quantile) Plot
Scale-Location
Residuals vs Leverage
Looking at the summary, it has p-value of 2.2e-16, which indicates that there is a highly statistically significant relationship between the two variables. So, why do we need to look at other things like residuals?
P-values by themselves can potentially be misleading without analyis of the residuals to ensure the model does not have any problems.
1.Residuals vs Fitted Plot: Residuals represent variation left unexplained by the model.Residual plots are used to look for underlying patterns in the residuals that may mean that the model has a problem.For a correct linear regression, the data needs to be linear so this will test if that condition is met.
2.Normal Q–Q (quantile-quantile) Plot Residuals should be normally distributed and the Q-Q Plot will show this. If residuals follow close to a straight line on this plot, it is a good indication they are normally distributed.For our model, the Q-Q plot shows pretty good alignment to the the line with a few points at the top slightly offset. Probably not significant and a reasonable alignment.
3.Scale-Location This plot test the linear regression assumption of equal variance (homoscedasticity) i.e. that the residuals have equal variance along the regression line. It is also called the Spread-Location plot. The residuals in our case have equal variance(occupy equal space) above and below the line and along the length of the line.
4.Residuals vs Leverage This plot can be used to find influential cases in the dataset. An influential case is one that, if removed, will affect the model so its inclusion or exclusion should be considered.An influential case may or may not be an outlier and the purpose of this chart is to identify cases that have high influence in the model. Outliers will tend to exert leverage and therefore influence on the model.An influential case will appear in the top right or bottom left of the chart inside a red line which marks Cook’s Distance. An example is shown below. The case on the right shows 4 item, in the red dashed line. Removing this from the dataset would have a significant affect on the model, which may or may not be desirable.
Linear Regression Prediction & Accuracy.
## [1] "RMSE 33.8595903089837"## [1] "Error rate 0.0544967670074829"## [1] "R2 0.999093423880329"Polynominal Regression
Polynomial regression is a special case of linear regression where we fit a polynomial equation on the data with a curvilinear relationship between the target variable and the independent variables.
In a curvilinear relationship, the value of the target variable changes in a non-uniform manner with respect to the predictor (s).
poly function returns or evaluates orthogonal polynomials of degree 1 to degree over the specified set of points x: these are all orthogonal to the constant polynomial of degree 0.
poly_reg<-lm( btc_market_price~
poly( btc_market_cap,2)+ poly( btc_hash_rate,2)+
poly( btc_difficulty,2)+ poly( btc_miners_revenue,2)+
poly( btc_estimated_transaction_volume_usd,2), data = subTrain1)
summary(poly_reg)##
## Call:
## lm(formula = btc_market_price ~ poly(btc_market_cap, 2) + poly(btc_hash_rate,
## 2) + poly(btc_difficulty, 2) + poly(btc_miners_revenue, 2) +
## poly(btc_estimated_transaction_volume_usd, 2), data = subTrain1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -148.28 -11.54 -4.62 14.30 265.60
##
## Coefficients:
## Estimate Std. Error t value
## (Intercept) 1.139e+03 3.568e-01 3191.369
## poly(btc_market_cap, 2)1 1.795e+05 3.158e+02 568.435
## poly(btc_market_cap, 2)2 1.135e+03 1.172e+02 9.681
## poly(btc_hash_rate, 2)1 -1.127e+04 2.833e+02 -39.798
## poly(btc_hash_rate, 2)2 8.981e+02 1.111e+02 8.080
## poly(btc_difficulty, 2)1 9.082e+03 3.307e+02 27.467
## poly(btc_difficulty, 2)2 -1.172e+03 1.142e+02 -10.258
## poly(btc_miners_revenue, 2)1 1.788e+04 2.301e+02 77.726
## poly(btc_miners_revenue, 2)2 -4.560e+03 8.601e+01 -53.019
## poly(btc_estimated_transaction_volume_usd, 2)1 -2.062e+03 1.208e+02 -17.063
## poly(btc_estimated_transaction_volume_usd, 2)2 5.619e+02 5.648e+01 9.950
## Pr(>|t|)
## (Intercept) < 2e-16 ***
## poly(btc_market_cap, 2)1 < 2e-16 ***
## poly(btc_market_cap, 2)2 < 2e-16 ***
## poly(btc_hash_rate, 2)1 < 2e-16 ***
## poly(btc_hash_rate, 2)2 7.89e-16 ***
## poly(btc_difficulty, 2)1 < 2e-16 ***
## poly(btc_difficulty, 2)2 < 2e-16 ***
## poly(btc_miners_revenue, 2)1 < 2e-16 ***
## poly(btc_miners_revenue, 2)2 < 2e-16 ***
## poly(btc_estimated_transaction_volume_usd, 2)1 < 2e-16 ***
## poly(btc_estimated_transaction_volume_usd, 2)2 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 26.27 on 5410 degrees of freedom
## Multiple R-squared: 0.9999, Adjusted R-squared: 0.9999
## F-statistic: 5.426e+06 on 10 and 5410 DF, p-value: < 2.2e-16Polynomial Regression Prediction & Accuracy
## [1] "RMSE 33.8595903089837"## [1] "Error rate 0.0544967670074829"## [1] "R Square 0.999682461536609"Spline Regression
- Spline is a special function defined piece-wise by polynomials. The term “spline” is used to refer to a wide class of functions that are used in applications requiring data interpolation and/or smoothing. The data may be either one-dimensional or multi-dimensional.
- Spline Regression is one of the non-parametric regression technique. In this technique the dataset is divided into bins at intervals or points which we called as knots. Also this bin has its separate fit.
- The disadvantages of the polynomial regression can be overcome by using Spline Regression. Polynomial regression only captures a certain amount of curvature in a nonlinear relationship. An alternative, and often superior, approach to modeling nonlinear relationships is to use splines.
- Splines provide a way to smoothly interpolate between fixed points, called knots. Polynomial regression is computed between knots. In other words, splines are series of polynomial segments strung together, joining at knots.
- The generic function quantile produces sample quantiles corresponding to the given probabilities.
- bs {splines} Generate the B-spline basis matrix for a polynomial spline.bs uses knots which is the internal breakpoints that define the spline.Typical values are the mean or median for one knot, quantiles for more knots.
knots <- quantile( subTrain1$btc_market_price, p = c( 0.25, 0.5, 0.75))
splinemodel<-lm( btc_market_price~
bs( btc_market_cap, knots = knots)+ bs( btc_hash_rate, knots = knots)+
bs( btc_difficulty, knots = knots)+ bs( btc_miners_revenue, knots = knots)+
bs( btc_estimated_transaction_volume_usd, knots = knots), data = subTrain1)Spline Regression Prediction & Accuracy
## [1] "RMSE 13.4968546288147"## [1] "Error rate 0.0217230904251439"## [1] "R Square 0.999810613324564"Generalized Linear Model
- Why Use GAM?Relationships between the individual predictors and the dependent variable follow smooth patterns that can be linear or nonlinear.
- Mathematically speaking, GAM is an additive modeling technique where the impact of the predictive variables is captured through smooth functions which—depending on the underlying patterns in the data—can be nonlinear.
- GAM can capture common nonlinear patterns that a classic linear model would miss.
- GAM framework allows us to control smoothness of the predictor functions to prevent overfitting. By controlling the wiggliness of the predictor functions, we can directly tackle the bias/variance tradeoff.
lmfit6 <- gam(btc_market_price ~ btc_estimated_transaction_volume_usd + btc_miners_revenue, data=bitcoin_dataset)
summary(lmfit6)##
## Family: gaussian
## Link function: identity
##
## Formula:
## btc_market_price ~ btc_estimated_transaction_volume_usd + btc_miners_revenue
##
## Parametric coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 2.834e-11 4.107e-13 69.0 <2e-16 ***
## btc_estimated_transaction_volume_usd 8.448e-07 4.667e-08 18.1 <2e-16 ***
## btc_miners_revenue 3.234e-04 4.686e-06 69.0 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## Rank: 2/3
## R-sq.(adj) = 0.974 Deviance explained = 97.4%
## GCV = 1.8968e+05 Scale est. = 1.8951e+05 n = 2258GAM Regression Prediction & Accuracy
## [1] "RMSE 241.540564283585"## [1] "Error rate 0.388757800507882"## [1] "R Square 0.958059200834039"Scatter Plot 3D Visualization
- Visualization part of gam model
- Diagnosing GAM model issues
##
## Method: GCV Optimizer: magic
## Model required no smoothing parameter selectionModel rank = 3 / 4Penalized Cubic Regression Spline
- Cubic Regression Spline is specified by bs=“cr”. These have a cubic spline basis defined by a modest sized set of knots spread evenly through the covariate values. They are penalized by the conventional intergrated square second derivative cubic spline penalty.
mod_lm4 <- gam(btc_market_price ~ s(btc_total_bitcoins, bs="cr")+s(btc_avg_block_size, bs="cr")+
s(btc_transaction_fees, bs="cr"),
data=bitcoin_dataset)
summary(mod_lm4)##
## Family: gaussian
## Link function: identity
##
## Formula:
## btc_market_price ~ s(btc_total_bitcoins, bs = "cr") + s(btc_avg_block_size,
## bs = "cr") + s(btc_transaction_fees, bs = "cr")
##
## Parametric coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1156.94 13.87 83.4 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Approximate significance of smooth terms:
## edf Ref.df F p-value
## s(btc_total_bitcoins) 8.994 9.000 454.74 <2e-16 ***
## s(btc_avg_block_size) 8.120 8.686 13.66 <2e-16 ***
## s(btc_transaction_fees) 4.930 5.552 215.07 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## R-sq.(adj) = 0.939 Deviance explained = 94%
## GCV = 4.3899e+05 Scale est. = 4.3451e+05 n = 2258- Diagnosing Penalized Cubic Regression Spline GAM model issues
##
## Method: GCV Optimizer: magic
## Smoothing parameter selection converged after 17 iterations.
## The RMS GCV score gradient at convergence was 0.5643143 .
## The Hessian was positive definite.
## Model rank = 28 / 28
##
## Basis dimension (k) checking results. Low p-value (k-index<1) may
## indicate that k is too low, especially if edf is close to k'.
##
## k' edf k-index p-value
## s(btc_total_bitcoins) 9.00 8.99 0.14 <2e-16 ***
## s(btc_avg_block_size) 9.00 8.12 0.91 <2e-16 ***
## s(btc_transaction_fees) 9.00 4.93 1.00 0.45
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Cubic Regression Spline Prediction & Accuracy
## [1] "RMSE 164.983422387511"## [1] "Error rate 0.265539631398429"## [1] "R Square 0.979740412173328"Extreme Gradient Boosting
XGBoost is an implementation of the Gradient Boosted Decision Trees algorithm.
XGBoost supports various objective functions, including regression, classification and ranking.
XGBoost gives among the best performances in many machine learning applications. It is optimized gradient-boosting machine learning library. The core algorithm is parallelizable and hence it can use all the processing power of your machine and the machines in your cluster. In R, according to the package documentation, since the package can automatically do parallel computation on a single machine, it could be more than 10 times faster than existing gradient boosting packages.
XGBoost shines when we have lots of training data where the features are numeric or mixture of numeric and categorical fields. It is also important to note that xgboost is not the best algorithm out there when all the features are categorical or when the number of rows is less than the number of fields (columns).
The data argument in the xgboost R function is for the input features dataset. It accepts a matrix, dgCMatrix, or local data file. The nrounds argument refers to the max number of iterations (i.e. the number of trees added to the model).
There are different hyperparameters that we can tune and the parametres are different from baselearner to baselearner. In tree based learners, which are the most common ones in xgboost applications, the following are the most commonly tuned hyperparameters:
learning rate: learning rate/eta- governs how quickly the model fits the residual error using additional base learners. If it is a smaller learning rate, it will need more boosting rounds, hence more time, to achieve the same reduction in residual error as one with larger learning rate. Typically, it lies between 0.01 - 0.3
The three hyperparameters below are regularization hyperparameters.
gamma: min loss reduction to create new tree split. default = 0 means no regularization.
lambda: L2 reg on leaf weights. Equivalent to Ridge regression.
alpha: L1 reg on leaf weights. Equivalent to Lasso regression.
max_depth: max depth per tree. This controls how deep our tree can grow. The Larger the depth, more complex the model will be and higher chances of overfitting. Larger data sets require deep trees to learn the rules from data. Default = 6.
subsample: % samples used per tree. This is the fraction of the total training set that can be used in any boosting round. Low value may lead to underfitting issues. A very high value can cause over-fitting problems.
colsample_bytree: % features used per tree. This is the fraction of the number of columns that we can use in any boosting round. A smaller value is an additional regularization and a larger value may be cause overfitting issues.
n_estimators: number of estimators (base learners). This is the number of boosting rounds.
xgbGrid <- expand.grid(nrounds = c(140,160), # this is n_estimators in the python code above
max_depth = c(10, 15, 20, 25),
colsample_bytree = seq(0.5, 0.9, length.out = 5),
## The values below are default values in the sklearn-api.
eta = 0.3,
gamma=0,
min_child_weight = 1,
subsample = 1
)
model_xgb <- train(btc_market_price ~ .,
data = subTrain,
method = "xgbTree",
preProcess = c("scale", "center"),
trControl = trainControl(method = "repeatedcv",
number = 5,
repeats = 3,
verboseIter = FALSE),
tuneGrid = xgbGrid,
verbose = 0)## [18:58:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:58:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [18:59:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:00:59] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:35] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:01:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:09] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:26] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:56] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:02:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:12] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:14] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:17] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:19] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:23] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:29] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:31] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:40] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:47] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:52] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:54] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:03:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:04] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:13] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:15] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:16] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:18] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:20] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:21] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:22] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:24] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:25] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:27] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:28] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:30] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:32] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:33] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:34] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:36] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:37] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:38] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:39] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:41] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:42] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:43] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:44] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:45] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:46] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:48] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:49] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:50] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:51] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:53] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:55] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:57] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:04:58] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:00] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:01] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:02] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:03] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:05] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:06] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:07] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:08] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:10] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.
## [19:05:11] WARNING: amalgamation/../src/objective/regression_obj.cu:174: reg:linear is now deprecated in favor of reg:squarederror.model_xgb$results ## eta max_depth gamma colsample_bytree min_child_weight subsample nrounds
## 1 0.3 10 0 0.5 1 1 140
## 3 0.3 10 0 0.6 1 1 140
## 5 0.3 10 0 0.7 1 1 140
## 7 0.3 10 0 0.8 1 1 140
## 9 0.3 10 0 0.9 1 1 140
## 11 0.3 15 0 0.5 1 1 140
## 13 0.3 15 0 0.6 1 1 140
## 15 0.3 15 0 0.7 1 1 140
## 17 0.3 15 0 0.8 1 1 140
## 19 0.3 15 0 0.9 1 1 140
## 21 0.3 20 0 0.5 1 1 140
## 23 0.3 20 0 0.6 1 1 140
## 25 0.3 20 0 0.7 1 1 140
## 27 0.3 20 0 0.8 1 1 140
## 29 0.3 20 0 0.9 1 1 140
## 31 0.3 25 0 0.5 1 1 140
## 33 0.3 25 0 0.6 1 1 140
## 35 0.3 25 0 0.7 1 1 140
## 37 0.3 25 0 0.8 1 1 140
## 39 0.3 25 0 0.9 1 1 140
## 2 0.3 10 0 0.5 1 1 160
## 4 0.3 10 0 0.6 1 1 160
## 6 0.3 10 0 0.7 1 1 160
## 8 0.3 10 0 0.8 1 1 160
## 10 0.3 10 0 0.9 1 1 160
## 12 0.3 15 0 0.5 1 1 160
## 14 0.3 15 0 0.6 1 1 160
## 16 0.3 15 0 0.7 1 1 160
## 18 0.3 15 0 0.8 1 1 160
## 20 0.3 15 0 0.9 1 1 160
## 22 0.3 20 0 0.5 1 1 160
## 24 0.3 20 0 0.6 1 1 160
## 26 0.3 20 0 0.7 1 1 160
## 28 0.3 20 0 0.8 1 1 160
## 30 0.3 20 0 0.9 1 1 160
## 32 0.3 25 0 0.5 1 1 160
## 34 0.3 25 0 0.6 1 1 160
## 36 0.3 25 0 0.7 1 1 160
## 38 0.3 25 0 0.8 1 1 160
## 40 0.3 25 0 0.9 1 1 160
## RMSE Rsquared MAE RMSESD RsquaredSD MAESD
## 1 39.03400 0.9997143 3.867351 25.023371 3.546794e-04 1.7577150
## 3 33.18108 0.9998120 3.425709 17.872421 1.829663e-04 1.3737375
## 5 37.31503 0.9997661 3.674326 19.179381 2.239330e-04 1.5042837
## 7 22.10033 0.9999162 2.489683 10.446471 8.835360e-05 0.9782250
## 9 19.15121 0.9999365 2.228535 9.277475 6.452859e-05 0.7787798
## 11 44.45382 0.9996732 4.240647 22.769227 3.019777e-04 1.5515580
## 13 29.11588 0.9998498 3.121364 16.054773 1.656704e-04 1.3285063
## 15 27.35755 0.9998652 2.875670 16.348729 1.340044e-04 1.4208245
## 17 20.72887 0.9999220 2.417198 12.214128 9.424477e-05 1.1041989
## 19 17.78226 0.9999508 2.127081 6.524760 3.807719e-05 0.6610867
## 21 42.59284 0.9996913 3.991861 22.418154 2.996915e-04 1.3100600
## 23 31.25821 0.9998335 3.172345 15.587641 1.466992e-04 1.2304602
## 25 28.70454 0.9998739 2.949386 10.537141 9.008602e-05 0.7973135
## 27 23.29094 0.9999056 2.660002 12.429602 9.178515e-05 1.1248811
## 29 19.26550 0.9999407 2.247684 7.132943 4.805357e-05 0.7821813
## 31 36.03812 0.9998017 3.798457 13.336798 1.265363e-04 0.9967262
## 33 27.92913 0.9998598 3.068617 15.623109 1.469525e-04 1.6367994
## 35 24.71652 0.9998876 2.767246 14.545188 1.309887e-04 1.2894564
## 37 20.78230 0.9999266 2.390776 9.009320 6.746089e-05 0.8252055
## 39 20.65820 0.9999333 2.252980 7.886065 5.210519e-05 0.7411809
## 2 39.03396 0.9997143 3.840191 25.023252 3.546726e-04 1.7560321
## 4 33.18124 0.9998120 3.401858 17.872200 1.829661e-04 1.3735135
## 6 37.31472 0.9997661 3.652922 19.179483 2.239312e-04 1.5032044
## 8 22.10016 0.9999162 2.469937 10.446723 8.835524e-05 0.9790382
## 10 19.15092 0.9999365 2.211328 9.277738 6.453012e-05 0.7808012
## 12 44.45381 0.9996732 4.240394 22.769204 3.019772e-04 1.5515495
## 14 29.11586 0.9998498 3.121129 16.054796 1.656708e-04 1.3285211
## 16 27.35753 0.9998652 2.875456 16.348726 1.340042e-04 1.4208145
## 18 20.72887 0.9999220 2.417110 12.214123 9.424471e-05 1.1041939
## 20 17.78226 0.9999508 2.126974 6.524756 3.807719e-05 0.6611015
## 22 42.59281 0.9996913 3.991856 22.418093 2.996899e-04 1.3100574
## 24 31.25817 0.9998335 3.172340 15.587609 1.466987e-04 1.2304575
## 26 28.70448 0.9998739 2.949379 10.537133 9.008597e-05 0.7973170
## 28 23.29092 0.9999056 2.659998 12.429597 9.178509e-05 1.1248812
## 30 19.26545 0.9999407 2.247677 7.132948 4.805362e-05 0.7821824
## 32 36.03809 0.9998017 3.798454 13.336753 1.265357e-04 0.9967265
## 34 27.92913 0.9998598 3.068615 15.623119 1.469526e-04 1.6368023
## 36 24.71647 0.9998876 2.767241 14.545150 1.309883e-04 1.2894575
## 38 20.78225 0.9999266 2.390771 9.009340 6.746094e-05 0.8252071
## 40 20.65812 0.9999333 2.252975 7.886036 5.210475e-05 0.7411819plot(model_xgb)
Extreme Gradient Boosting Prediction & Accuracy
## MSE: 80.92633 MAE: 5.168907 RMSE: 8.995907
## [1] "RMSE 8.99590656624651"## [1] "Error rate 0.0133612673135272"## [1] "R Square 0.999954333588047"Compare Regression Models
## 'data.frame': 1 obs. of 4 variables:
## $ Algorithm: chr "Linear Regression"
## $ RMSE : num 33.9
## $ R2 : num 0.999
## $ Error : num 0.0545## 'data.frame': 1 obs. of 4 variables:
## $ Algorithm: chr "Polynomial Regression"
## $ RMSE : num 33.9
## $ R2 : num 1
## $ Error : num 0.0545## 'data.frame': 1 obs. of 4 variables:
## $ Algorithm: chr "Spline Regression"
## $ RMSE : num 13.5
## $ R2 : num 1
## $ Error : num 0.0217## 'data.frame': 1 obs. of 4 variables:
## $ Algorithm: chr "GAM Regression"
## $ RMSE : num 242
## $ R2 : num 0.958
## $ Error : num 0.389## 'data.frame': 1 obs. of 4 variables:
## $ Algorithm: chr "Cubic Regression Spline"
## $ RMSE : num 165
## $ R2 : num 0.98
## $ Error : num 0.266## 'data.frame': 1 obs. of 4 variables:
## $ Algorithm: chr "Extreme Gradient Boosting"
## $ RMSE : num 9
## $ R2 : num 1
## $ Error : num 0.0134Conclusion
As such, we provide evidence suggesting that technical analysis is useful in a market like bitcoin whose value is mainly driven by by fundamental factors. Extreme Gradient Boosting outperforms other model with lesser error and with R2=1 the model completely fit.